Many video encoding applications utilize statistical multiplexing techniques to combine several compressed video bit streams into a single multiplexed bit stream, e.g., for transmission on a single channel. The bit rate of a given compressed stream generally varies with time based on the complexity of the corresponding video signals. A statistical multiplexer attempts to estimate the complexity of the various video frame sequences of a video signal and allocates channel bits among the corresponding compressed video bit streams so as to provide an approximately constant level of video quality across all of the multiplexed streams. For example, a given video frame sequence with a relatively large amount of spatial activity or motion may be more complex than other sequences and therefore allocated more bits than the other sequences.
An example of a statistical multiplexing encoding system is described in M. Perkins & D. Arnstein, Statistical Multiplexing of Multiple MPEG-2 Video Programs in a Single Channel, SMPTE J., vol. 104, no.9, p. 596-599, September, 1995. As described in this reference, multiple encoders each receive a respective program, encode the program, and place their compressed picture data of a video signal of the program in a corresponding buffer of fixed size pending submission to a multiplexer. (As per MPEG-2 parlance, a "program" is a collection of one or more related signals. Herein, a program is presumed to include a video signal but may also include one or more associated audio signals, a close caption text signal, etc.) A multiplexer receives the encoded programs from the multiple encoders, in the form of a bit stream. A different bit rate may be assigned to each bit stream depending on a respective estimate of the number of bits needed by the video bit stream of the corresponding program to achieve the same level of quality as the other programs with which it is multiplexed. (Generally speaking, the bit rate of the video bit stream of a program is variable whereas the bit rates of the audio and other associated bit streams of a program are constant. This invention is illustrated in the context of adjusting only the bit rate of the video bit stream portion of a program.) On the receiving end of the communication, a decoder receives the multiplexed multiple program bit stream and discards the data it does not need (e.g., if the decoder is a set top box, only the "tuned" or selected program is retained, whereas the data of each non-selected program is discarded). The retained data is inputted to the decoder's input buffer of a fixed size pending decoding. The removal of data from the decoder buffer for decoding is controlled in a strict fashion to effect a constant end-to-end delay for any selected program. In the statistical multiplexing encoding scheme, the relative timing of each to-be-multiplexed program is independent. A first encoder for a first program may have many pictures of compressed data in its buffer pending submission to the multiplexer while a second encoder for a second bit stream may have only a few pictures in its buffer pending submission to the multiplexer. A decoder that selects the first program for decoding will be receiving "earlier" pictures than a decoder that selects the second program for decoding. Such a variable delay is eliminated by each decoder lengthening or shortening the amount of time the received pictures spend in the decoder buffer pending decoding to effect the above-noted constant end-to-end delay.
FIG. 1 shows a conventional statistical multiplexer 10. The statistical multiplexer 10 includes a number n of video sources 12-i, i=1, 2, . . . n. Each of the n video sources 12-i supplies a video signal (e.g., a video bit stream) to a corresponding encoder 14-i. The encoders 14-i generate compressed video bit streams which are supplied to a multiplexer 16. The multiplexer 16 combines all of the compressed video bit streams into a single multiplexed bit stream. Each of the encoders 14-i sends statistics about the video bit stream that it is encoding to a statistics computer 18. The statistics computer 18 uses the statistics received from the encoders 14-i to determine a suitable allocation of available channel bits among the n video bit streams. The statistics computer 18 sends information regarding the allocated bit rate of each video bit stream to the corresponding encoders 14-i.
A significant problem with the conventional statistical multiplexer 10 is that bit allocation decisions are generally made using only a posteriori statistics. The statistics computer 18 therefore only receives information about pictures of the n video bit streams that have already been encoded, and must use this information to allocate bits among those pictures that have yet to be encoded. This reliance on a posteriori statistics can result in periods of poor video quality when the video bit streams are eventually demultiplexed, decoded and displayed. The degradation in video quality is particularly severe if one or more of the n to-be-encoded video signals includes a relatively large number of transitions between very complex pictures and very simple pictures. Such transitions can lead the statistics computer 18 to allocate an excessive number of bits to simple pictures of a given video bit stream, or to allocate an insufficient number of bits to complex pictures of the video bit stream, resulting in substantial degradation in perceived video quality for one or more of the multiplexed video bit streams.
The Perkins and Arnstein reference suggests an alternative to the a posteriori statistical multiplexing scheme which uses a priori statistics. An a priori statistical multiplexer is shown in FIG. 2. In the system 40, each of n video sources 42-i for i=1, 2, . . . , n outputs a to-be-encoded program to a corresponding statistics gatherer circuit or processor 43-i for i=1, 2, . . . , n, respectively. The statistics gatherers 43-i measure various statistics on the video signal of the program in order to determine the level of difficulty, at that particular instant, in compressing the uncompressed program. A statistics computer 48 receives the statistics gathered by each statistics gatherer 43-i, and allocates a bit rate to each encoder 44-i for i=1, 2, . . . , n based on the gathered statistics. A multiplexer 46 multiplexes, i.e., combines, the compressed program bit stream to form a single multiplexed bit stream for transmission on a single channel.
As noted by the Perkins and Arnstein reference, it is difficult to find statistic metrics which lead to an appropriate allocation of bit rates. For instance, if the statistic metric employed by a statistics gatherer significantly underestimates the needs of the program for which it gathers statistics, then the video bit stream of that particular program will be encoded at a significantly lower quality level relative to the other programs. This disparity in quality level will only be corrected when the nature of the video bit stream of that program changes such that the statistics gathered for that program more accurately reflect its encoding needs.
L. Wang & A. Vincent, Joint Rate Control for Multi-Program Video Coding, IEEE TRANS. ON CONS. ELECS., vol. 42, no. 3, August, 1996, proposes an alternative to statistical multiplexing. According to the Wang and Vincent reference, multiple programs are encoded as a single program. A super frame is defined including one frame from the video signal of each to-be-multiplexed program. A bit target is set for each super frame and the bit target may be nearly met by determining a quantization level for the super frame that will generate approximately the correct number of bits for each constituent frame in the super frame. The quantization level is identified by encoding each constituent frame of the super frame several times at different quantization levels, determining the number of bits generated for each encoded frame, and then searching over the encoded frames thus produced using a binary search algorithm. Note that in this encoding technique, each program is synchronized and the timing is controlled for all programs as if a single encoder produced all of the compressed bit streams. This means that the relative frame transmission rates of each program (in compressed form) do not vary relative to each other. This simplifies the rate control and is an appropriate technique to use when the receiver must decode each of the programs multiplexed together.
Note that the decoder disclosed in this reference is intended to receive and decode each program and must therefore have a large enough buffer to accommodate the frames of each program. However, even if the decoder selects only a single program for decoding and discards data for the other programs, the decoder buffer must still be increased in size relative to a decoder buffer that receives a statistically multiplexed signal. Consider that the synchronization requirement on the encoders causes each super frame to include a frame of each of the programs. However, the size of each frame varies depending on the frame type, its complexity, etc. Suppose a decoder selects a q.sup.th program for decoding. The j.sup.th frame for program q might be separated from the (j+1).sup.th frame by several large frames for other programs 1,2, . . . q-1. Therefore, the decoder buffer must be increased to store more frames (e.g., j-2, j-1, j) to ensure that the decoder does not underflow while waiting for the (j+1).sup.th frame. Such a problem does not exist in a statistical multiplexer since there is no requirement to multiplex one frame from each of the other programs between frames j and j+1, especially, when such frames are very large. Any increase in decoder buffer size requirements presents a particularly large expense for a video distribution network such as a direct broadcast satellite network or a cable television network wherein a single multiplexer may transmit a multiplexed signal to thousands or even millions of set top box receiver/decoders. As such, the statistical multiplexing technique, wherein the multiplexer allocates bit rates but wherein each program has independent timing, is preferred.
As is apparent from the above, there is a need for improved statistical multiplexing which is better able to maintain quality across multiple video bit streams.